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Abstract 



Background’. In today’s society it is essential to be able to read fluently, particularly, expository 
and argumentative texts (Chambliss, 1995; Gresten, Fuchs, Williams, & Baker, 2001). Accrued 
research indicates that critical reading of argumentative text is important not only for succeeding 
on high-school and college assignments (such as critical analysis of literary text, debates, and 
research papers) but also for making real-life decisions (for example, from buying consumer 
products to electing apolitical candidate; Knudson, 1992; Larson, Britt, & Larson, 2004). Further, 
McCann (1989) stated, “Argument is an essential instrument for a free society that deliberates 
about social, political and ethical issues” (as cited in Knudson, 1992; p. 170). Despite the 
importance of comprehending argumentative text, the research indicates that extended arguments 
in persuasive essays are not easily comprehended by students (Chambliss, 1995). 

The researchers have indicated several reasons that explain this difficulty in comprehending 
expository or argumentative text (Sanez & Fuchs, 2001; Chambliss, 1995). One of the reasons for 
poor understanding of argumentative text is that often students are not exposed to reading 
materials that include extensive arguments (Chambliss, 1995). A second reason is that 
argumentative text is inherently difficult because unlike narrative text, argumentative text is often 
embedded in other genre (i.e., informational or narrative text), and thus, it is difficult for the 
students to navigate through the text to figure out the author’s argument. A third reason is that 
often a reader has to simultaneously juggle many skills to fully comprehend an author’s argument. 
Some of those skills are: (a) ability to recognize the author’s argument (i.e., author’s claim, 
supporting reasons, and evidence); (b) ability to bracket biases or preconceived notions and read 
the author’s argument with an open mind; and (c) ability to evaluate, question and critique written 
arguments (i.e., is the reason relevant, truly supporting the position, what makes it a good reason, 
etc.). In other words, young students believe that most written texts are true and fail to question 
the author’s argument. 

Although research has investigated persuasive writing instruction, there has been a lack of 
instructional research on reading argumentative text that specifically addresses the development of 
critical and analytical understanding of written arguments in school-age children. In short, 
comprehension of argumentative text is highly complex, and there is a need for research on 
instruction in how to critically analyze written arguments. Thus, the purpose of the proposed 
presentation is to present the findings about the effectiveness of teaching students with reading 
difficulties a genre-specific strategy for understanding and analyzing written arguments. 

Purpose of study: The purpose of this research was to examine the effectiveness of teaching fifth 
grade students with reading difficulties a genre- specific strategy for summarizing and critically 
analyzing written arguments. In addition, this research explored whether learning this particular 
reading strategy informed the students’ ability to write effective and convincing persuasive essays. 
The researcher developed reading comprehension strategy named critical ancdysis of 
argumentative text (CAAT) that guided students in identifying the structural elements of written 
argument, summarizing, and critically analyzing the content of the argument. The instruction was 
intended to enhance students’ analytical and critical reading skills. 
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Setting/ Participants: A total of seven fifth grade students with reading comprehension difficulties 
from the inner city school in the Mid- Atlantic region participated in the study. The reading 
specialist identified ten students who had difficulties in reading comprehension. Further, the 
students were screened for comprehension risks. Out of ten students, eight students scored at 
instructional level (i.e., between 90% and 97%) on oral reading passage and frustration level for 
comprehension questions (i.e., 60% or below) for fourth grade level text from Qualitative Reading 
Inventory (Leslie & Caldwell, 2005). Initially there were eight participants and they were divided 
into four pairs for instructional purposes. However, one student was dropped out of the study 
because of the behavior issues. Thus, the fourth pair was dissolved and one student from the fourth 
pair was instructed along with the third pair. Each pair received genre- specific comprehension 
instruction for 35 to 40 minutes three times per week for seven weeks. 

Intervention : The Critical Analysis of Argumentative Text ( CAAT) strategy used in this 
investigation is influenced by Chambliss’ (1995) comprehension model. The Chambliss’ (1995) 
model facilitated readers to comprehend written arguments and it consisted of three stages: (a) 
recognizing an argument; (b) identifying elements of argument (i.e., claim and evidence); and (c) 
constructing the gist of a written argument (Chambliss, 1995). However, in this investigation, the 
reading comprehension strategy CAAT not only guided students to identify elements of argument, 
but it also helped them to summarize and critically evaluate written arguments. In short, this 
strategy was an attempt to combine underlying aspects of comprehension (i.e., identifying, 
summarizing, and analyzing) and structural elements of argumentative text. 

The CAAT strategy was divided into six major steps: (a) Step 1: focused on setting the purpose for 
reading argumentative text and bracketing biases; (b) Step 2: read the text carefully; (c) Step 3: 
identified, underlined and labeled parts of the argument (e.g., author’s position on a topic, reasons, 
evidence, opposite view, rebuttal and conclusion); (d) Step 4: focused on summarizing the 
argument (i.e., including important elements and excluding minor details); (e) Step 5: aimed at 
analyzing author’s overall argument (e.g., whether the author’s reasons are strong; whether the 
author’s argument is convincing); and (f) Step 6: asked readers’ to present their views and 
supporting reasons on a given topic. 

The CAAT strategy instruction was comprised of 21 sessions. Each session lasted for 35 to 40 
minutes and they were divided into following ways: (a) Sessions 1-4: focused on developing 
background knowledge for persuasive genre, differences between persuasion and argumentation, 
bracketing biases and other related topics; (b) Sessions 5-7-: emphasized on demonstrating, 
practicing, and reviewing CAAT strategy; (c) Sessions 8-18: aimed at teaching each step of the 
strategy with variety of examples/non-examples, provided practice to master each step, and 
focused on in-depth dialogic discourse about the argumentative text; and (d) Sessions 19-21: 
intended to bring it altogether for the readers and provided ample practice to master identifying, 
summarizing and analyzing skills. 

The investigator taught the CAAT strategy using strategy instruction and self-evaluation 
procedures that were based on Graham and Harris’s (1989; 1999) suggestions for effective 
strategy instruction. In addition, the design of the instruction also drew upon ideas from Beck and 
colleagues “Question the Author” approach (Beck, McKeown, Hamilton, & Kucan, 1997). 
Furthermore, the investigator utilized scripted lesson plans, self-evaluation forms, experimenter- 
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created persuasive essays, examples/non-examples to amplify the parts of argument, handouts, 
activity packets, and question cue cards to support students’ learning. During instructional 
sessions, the investigator explained and demonstrated how to utilize the CAAT strategy to 
identify, summarize, and critically analyze an argument. In short, the CAAT comprehension 
strategy instruction was structured, systematic and utilized visual teaching aids to improve 
students’ understanding about argumentative text. 

Research Design : The proposed investigation utilized a multiple baseline design with multiple 
probes across paired participants. The single subject design involved several phases such as 
baseline, intervention, posttest, and maintenance. In addition, the investigator included pre- 
baseline phase to briefly review parts of argument, persuasive text, and assessment procedures, in 
case the students were unfamiliar with persuasive genre. The baseline data was compared to 
posttest data of each individual student and thus the design itself “demonstrated experimental 
control using one person as both the control and experimental participant” (Kenndy 2005). The 
criterion measure (i.e., Argument Assessment Analysis) was administered to measure students’ 
ability to summarize and critically analyze the written arguments. However, the summary score 
were considered to establish a stable baseline. In this design, the effects of instruction are 
demonstrated when each pair maintains a stable baseline and improves only after instruction is 
provided. 

Data Collection'. Reading Comprehension Assessment: Argument Analysis Assessment (AAA). An 
experimenter-created curriculum-based assessment was individually administered in a school 
setting. Each student was asked to read argumentative text, while reading they could take notes or 
underline parts of argument. Then, they were asked to verbally summarize (i.e., include important 
parts of argument and exclude minor details) and analyze (i.e., state whether overall argument was 
convincing or not. Why?) the written arguments. The student’s summary and critical analysis of 
texts were audio taped for scoring and analysis purposes. For the assessment, the investigator 
developed 12 controlled argumentative text (such as “Should Pluto be a planet?”) at 5 th grade level 
of 250 to 350 words that included parts of argument (i.e., author’s position, reasons, evidence, 
opposing position, rebuttal and conclusion). The argumentative texts were utilized as an 
assessment probe during baseline, posttest and maintenance phase. 

The analytical scoring procedures used for analyzing summaries were influenced by Midgette, 
Haria, and MacArthur (2008). It rated the important elements of argument (position, reason, 
evidence, opposing position, and rebuttal) that were included in a summary. Each element score 
ranged from zero-to-two (0- no response and 2-complete, clear and accurate response) except 
evidence score ranged from (0- no response and 1- include relevant information). The student 
summaries received maximum score of 12 points and minimum of zero. In addition, the 
investigator separately counted non-functional units (i.e., minor details, irrelevant information, 
description, extra information) that were present in the summary. The non-functional units weaken 
the summary. Therefore, lower the number of non-functional units, better is the summary. Further, 
the investigator also examined the quality of summary. The overall summary score ranged from 
zero-to-three (0- irrelevant response; 3- clear, and succinct summary with few or no minor details). 

Further, the scoring procedures developed for critical analysis aspect of reading comprehension 
were based on pre-defined criteria for arguments that were partially influenced by Ferretti, 
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MacArthur, & Dowdy, (2000), Grice’s (1975) Cooperative Principle and maxims (i.e., the 
principle of clarity, honesty, efficiency and relevance), and Blair and Johnson’s (1987) three 
criteria (i.e., relevance, acceptance and sufficiency). Overall scoring guide was created to capture 
how students critiqued and analyzed a given argumentative text. The scores ranged from zero-to- 
four (0-no response/irrelevant response; 4- fully developed response with elaborate explanation). 
The students received maximum of four points and minimum of zero. 

In addition, the investigator administered another part of reading comprehension known as the 
Identification of Elements in Written Argument (IEWA). It measured students’ ability to accurately 
identify and explain important parts of written arguments. This measure was administered only 
once during each phase to avoid practice effect. The purpose of administering this test was to 
understand whether students could successfully identify parts of argument and explain their 
reasons. The scoring procedures for this part considered correct (1 point) or incorrect (0 point) 
identification of parts of argument and the scoring for explanation ranged from Zero-to-two (0- no 
response; 2- correctly/fully explained) for each element. 

Persuasive Writing Assessment. Students wrote persuasive essays in response to prompts. The 
prompts and scoring procedures are modeled on previous research (Ferretti, MacArthur, & 

Dowdy, 2000; Midgette, Haria, & MacArthur, 2008). The students were asked to plan and write a 
persuasive essay on a given persuasive writing prompt. The students will write two essays during 
baseline and two during the posttest phase. The topics were randomly selected from the pool of 12 
topics. The students’ essays were rated on a 7-point primary trait scale for overall persuasiveness 
1 1 writing (ranging from 0- no response/irrelevant response; 7- fully elaborate persuasive essay). In 
addition, the scorers rated the essays using analytic scales called Elements of Persuasive Discourse 
for structural elements of persuasive discourse content (position, reasons, evidence, coherence) 
and audience awareness (opposing position, rebuttal) aspect of the essay. The score for each 
persuasive element ranged from zero-to-two. The content part fetched total of 12 points and 
audience awareness fetched four points. In all, the students may receive 16 points. 

The pre-post metacognitive interview was conducted to understand whether students are aware 
about their own reading and writing processes, skills and attitudes. The responses were typed and 
common themes were categorized and coded. The data for interview will be ready for the 
presentation. 

Data Analysis. The reading comprehension tests were transcribed by first author and scored by 
two trained retired schoolteachers. Similarly the written essays were typed by first author and 
scored by two trained schoolteachers. The author graphed the data and visually analyzed the data 
to determine the effects of intervention on students’ ability to summarize and analyze written 
arguments. 

Findings: Reading Comprehension Scores. Analytic Summary Scores. All seven students made 
gains in analytical scores for reading summary from baseline to posttest (See Table 1). Student 
established stable baseline with mean scores ranging from 2.1 to 4.3. All posttest scores were 
higher than all baseline scores with mean scores ranging from 9.3 to 1 1.3. Thus, the percent of 
non-overlapping data (PND) was 100%. (PND is often considered in multiple baseline design as a 
measure of overall magnitude of effects analogous to effect size in group designs.) Scores were 
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maintained at one month for first two pairs. At two months, scores for the first pair declined 
somewhat but remained much above baseline performance. 

Overall Quality Scores for Summary. All students made gains in overall quality (see Table 1). The 
mean scores for baseline for individual students ranged from 0.0 to 0.3 on a scale from 0 to 3. The 
student made substantial progress from baseline to posttest. The mean gains in overall quality for 
posttest ranged from 1.8 to 3.0. At one month interval, the first two pair maintained their scores. 
However, at two months interval, there was a slight drop in overall score for first pair. But the 
scores in second maintenance phase were still higher than the baseline. 

Overall Scores for Critical Analysis. All seven students showed progress on overall critical 
analysis measure (see Table 1). Gains in mean scores from baseline to posttest for individual 
students ranged from 1.0 to 3.3 on a scale from 0 to 4. PND for critical analysis was 100%. These 
results suggest that a well integrated comprehension strategy was quiet effective for at risk fifth 
grade students on their ability to summarize and analyze complex argumentative text. 

Writing Scores. Elements of Persuasive discourse. Six students improved on analytical scores 
from baseline to posttest. Only, Quincy’s scores dropped from baseline (9.5) to posttest (6.2). 
Gains in mean score from baseline to posttest for the other six students ranged from 4.5 to 15.5 on 
a scale from 0- 16 (see Table 2). Though, the students were not provided direct instruction in 
writing effective persuasive essays, most students improved on elements of persuasive discourse 
scale. 

Primary Trait Sccdefor Overall Persuasiveness. The scores did not reflect gains from baseline to 
posttest. Mary and Jessica showed progress from baseline to posttest. On the other hand, Tony and 
Sean scores did not change from baseline to posttest. Annie and Quincy declined in their scores 
from baseline to posttest (see Table 2). For writing scores, it was understandable that students did 
not make substantial gains as the investigator did not teach how to write persuasive essay and also 
did not point out the connection between reading and writing. 

Conclusions’. The results indicate that genre-specific comprehension strategy improved students’ 
ability to summarize and critically analyze complex argumentative texts. Results are particularly 
noteworthy given that substantial gains were made with modest levels of instruction. The strategy 
instruction was implemented only 3 times a week for 35 minutes and the students were exposed to 
complex reading comprehension task of identify, summarizing and analyzing complex 
argumentative text. Overall results support that if students are given opportunity to explicitly learn 
comprehension strategies, engage in meaningful dialogical discourse to understand different 
perspectives, and critically analyze argumentative text, then they may develop a deeper 
understanding of the text. 
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Appendix B. Tables and Figures 



Table 1 



Mean Scores by Phase for Reading Summary and Critical Analysis 



Student 

Phase (# Text) 


Analytical Score 
for Summary 


Overall Score 
for Summary 


Overall Score for 
Critical Analysis 


Tony 


Baseline (3) 


2.7 


0.0 


1.0 


Posttest (3) 


9.3 


1.8 


2.6 


Maintenance (2) 


7.0 


1.0 


2.5 


Annie 


Baseline (3) 


4.3 


0.3 


1.0 


Posttest (3) 


10.0 


2.0 


1.7 


Maintenance (2) 


9.5 


2.5 


2.0 


Mary 


Baseline (4) 


3.5 


0.2 


1.0 


Posttest (3) 


11.3 


3.0 


3.3 


Maintenance (2) 


11.2 


2.7 


2.7 


Jessica 


Baseline (4) 


3.0 


0.2 


1.5 


Posttest (3) 


11.3 


3.0 


3.3 


Maintenance (2) 


11.2 


3.0 


2.7 


Clifford 


Baseline (5) 


2.2 


0.0 


1.7 


Posttest (3) 


9.5 


2.0 


2.8 


Sean 


Baseline (5) 


2.1 


0.2 


1.3 


Posttest (3) 


11.3 


3.0 


1.7 


Quincy 


Baseline (5) 


4.1 


0.2 


1.4 


Posttest (3) 


9.6 


2.0 


2.3 



2010 SREE Conference Abstract Template 



B-l 




Table 2 



Mean Scores by Phase for Elements of Persuasive discourse and Primary Trait Scale 



Student 

Phase (# Text) 


Elements of Persuasive 
Discourse 
(Written Essay) 


Primary Trait 
Scale 

(Written Essay) 


Tony 


Baseline (2) 


4.5 


2 


Posttest (2) 


6.2 


2 


Annie 


Baseline (2) 


9.5 


4.2 


Posttest (2) 


11.5 


4 


Mary 


Baseline (2) 


5 


1.5 


Posttest (2) 


10.5 


3.2 


Jessica 


Baseline (2) 


7.5 


3 


Posttest (2) 


15.5 


6 


Clifford 


Baseline (2) 


6 


3.2 


Posttest (2) 


10 


4.5 


Sean 


Baseline (1)* 


4.5 


4 


Posttest (2) 


11 


4 


Quincy 


Baseline (2) 


9.5 


3.2 


Posttest (2) 


6.2 


2.5 



Note: *Sean ’s baseline essay is missing. 
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